Papers with machine learning tasks
Augmenting Transformers with KNN-Based Composite Memory for Dialog (2021.tacl-1)
Copied to clipboard
| Challenge: | Recent work has focused on learning architectures with large memories capable of storing external knowledge. |
| Approach: | They propose a method to augment generative Transformer neural networks with information fetching modules. |
| Outcome: | The proposed approach improves performance in generative dialog modeling . external knowledge is retrieved from Wikipedia, images, and human-written dialog utterances . |
DropMix: A Textual Data Augmentation Combining Dropout with Mixup (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to overcome overfitting in text learning do not consider dimensionality . dimensionalization is important for deep neural networks to overcome the problem . |
| Approach: | They propose a saliency map-based approach to overcome overfitting in text learning . they propose augmentation regularization methods such as Dropout and Mixup to improve regularization . |
| Outcome: | Empirical results show that the proposed approach overcomes overfitting in text learning . dropout and mixup methods are effective in enhancing regularization . |
Indra: A Word Embedding and Semantic Relatedness Server (L18-1)
Copied to clipboard
Juliano Efson Sales, Leonardo Souza, Siamak Barzegar, Brian Davis, André Freitas, Siegfried Handschuh
| Challenge: | Word embedding/distributional semantic models are a fundamental component in many natural language processing (NLP) architectures. |
| Approach: | They propose a multi-lingual word embedding/distributional semantics framework which supports creation, use and evaluation of word embedded models. |
| Outcome: | The proposed tool supports the creation, use and evaluation of word embedding models. |
FeRG-LLM : Feature Engineering by Reason Generation Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | FeRG-LLM is a large language model that performs feature engineering at an 8billion-parameter scale. |
| Approach: | They propose a framework to perform feature engineering at an 8billion-parameter scale using conversational dialogues. |
| Outcome: | The proposed framework outperforms Llama 3.1 70B and Llma 3.2 on most datasets while using fewer resources and achieving reduced inference time. |
Multi-Modal Retrieval For Large Language Model Based Speech Recognition (2024.findings-acl)
Copied to clipboard
Aditya Gourav, Jari Kolehmainen, Prashanth Shivakumar, Yile Gu, Grant Strimel, Ankur Gandhe, Ariya Rastrow, Ivan Bulyko
| Challenge: | kNN-LM and cross-attention techniques are used to extend text based retrieval to other modalities . wide adoption of large language models has driven new application areas leveraging this technology . |
| Approach: | They propose to use kNN-LM and cross-attention techniques to extend text retrieval methods to other modalities. |
| Outcome: | The proposed methods outperform text-based retrieval and improve word error rate on a speech recognition dataset. |
A Corpus for Automatic Readability Assessment and Text Simplification of German (2020.lrec-1)
Copied to clipboard
| Challenge: | Using monolingual-only data, we can automate readability assessment and text simplification of simplified language. |
| Approach: | They present a corpus for automatic readability assessment and automatic text simplification for German using parallel and monolingual data. |
| Outcome: | The proposed corpus is compiled from web sources and contains information on text structure, typography, font style, and images. |
bgGLUE: A Bulgarian General Language Understanding Evaluation Benchmark (2023.acl-long)
Copied to clipboard
Momchil Hardalov, Pepa Atanasova, Todor Mihaylov, Galia Angelova, Kiril Simov, Petya Osenova, Veselin Stoyanov, Ivan Koychev, Preslav Nakov, Dragomir Radev
| Challenge: | bgGLUE is a benchmark for evaluating language models on natural language understanding (NLU) tasks in Bulgarian. |
| Approach: | They propose to use a benchmark to evaluate language models on NLU tasks in Bulgarian. |
| Outcome: | The proposed model performs well on sequence labeling tasks, but there is room for improvement for tasks that require more complex reasoning. |
Transkimmer: Transformer Learns to Layer-wise Skim (2022.acl-long)
Copied to clipboard
| Challenge: | Prior work has proposed to augment Transformer model with the capability of skimming tokens to improve its computational efficiency. |
| Approach: | They propose to add a parameterized predictor before each layer that learns to make the skimming decision. |
| Outcome: | The proposed model achieves 10.97x speedup on GLUE benchmark compared with BERT-base baseline with less than 1% accuracy degradation. |
On Efficient Retrieval of Top Similarity Vectors (D19-1)
Copied to clipboard
| Challenge: | Existing representation learning methods such as Word2vec represent word embeddings in the semantic space. |
| Approach: | They propose an efficient method for searching vectors via a non-metric matching function: inner product. |
| Outcome: | Experiments on data representations learned for different machine learning tasks show the proposed method outperforms existing methods. |
Wronging a Right: Generating Better Errors to Improve Grammatical Error Detection (D18-1)
Copied to clipboard
| Challenge: | grammatical error correction is a labor-intensive task that requires large amounts of training data. |
| Approach: | They propose to use a human-annotated corpus of human-generated grammatical errors to generate a synthetic model. |
| Outcome: | The proposed method outperforms the current state of the art in grammatical error correction . human annotators achieve 39.39 F1 scores, suggesting the model generates mostly human-like instances . |
Hausa Visual Genome: A Dataset for Multi-Modal English to Hausa Machine Translation (2022.lrec-1)
Copied to clipboard
Idris Abdulmumin, Satya Ranjan Dash, Musa Abdullahi Dawud, Shantipriya Parida, Shamsuddeen Muhammad, Ibrahim Sa’id Ahmad, Subhadarshi Panda, Ondřej Bojar, Bashir Shehu Galadanci, Bello Shehu Bello
| Challenge: | Hausa is considered a low resource language in natural language processing due to lack of resources. |
| Approach: | They propose a dataset that contains the description of an image in Hausa and its equivalent in English. |
| Outcome: | The Hausa Visual Genome is the first dataset of its kind . it can be used for Hausa-English machine translation, multi-modal research, image description . |
OPT-BENCH: Evaluating the Iterative Self-Optimization of LLM Agents in Large-Scale Search Spaces (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities in reasoning and tool use, but their ability to continuously refine solutions in response to dynamic environmental feedback remains underexplored. |
| Approach: | They propose a benchmark to evaluate self-improvement capabilities in large-scale search spaces by combining 20 machine learning tasks with 10 classic NP-hard problems. |
| Outcome: | The proposed framework emulates human-like cognitive adaptation and operates via a general perception–memory–reasoning loop, iteratively refining solutions based on environmental feedback. |
Why Voice Biomarkers of Psychiatric Disorders Are Not Used in Clinical Practice? Deconstructing the Myth of the Need for Objective Diagnosis (2024.lrec-main)
Copied to clipboard
| Challenge: | Anxiety and depression are the most prevalent mental disorders, affecting 3.9% and 3.6% of the world's population . |
| Approach: | They propose to shift the estimation of diagnoses towards estimation of clinical symptoms and signs, which address the limitations raised against diagnosis estimation. |
| Outcome: | The proposed paradigm shift will empower the use of vocal biomarkers in clinical practice. |